Systems and methods for medical image style transfer using deep neural networks

ABSTRACT

The current disclosure provides for mapping medical images to style transferred medical images using deep neural networks, while maintaining clinical quality of the style transferred medical image, thereby enabling a clinician to evaluate medical images in a preferred style without loss of clinically relevant content. In one embodiment the current disclosure provides for a method comprising, acquiring a medical image of an anatomical region of a subject, wherein the medical image is in a first style, selecting a target style, wherein the target style is distinct from the first style, selecting a clinical quality metric, selecting a trained style transfer network based on the target style and the clinical quality metric, mapping the medical image to a style transferred medical image using the trained style transfer network, wherein the style transferred medical image is in the target style, and displaying the style transferred medical image via a display device.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a Divisional Application of U.S. patent applicationSer. No. 16/752,461, entitled “SYSTEMS AND METHODS FOR MEDICAL IMAGESTYLE TRANSFER USING DEEP NEURAL NETWORKS”, filed Jan. 24, 2020, whichis herein incorporated by reference in its entirety.

TECHNICAL FIELD

Embodiments of the subject matter disclosed herein relate to medicalimaging, and more particularly, to systems and methods for mappingmedical images to a target style domain using deep neural networks.

BACKGROUND

Image processing devices are often used to obtain internal physiologicalinformation of a subject, such as a patient. For example, an imageprocessing device may be used to obtain images of the bone structure,the brain, the heart, the lungs, and various other anatomical regions ofa patient. Image processing devices may include magnetic resonanceimaging (MRI) systems, computed tomography (CT) systems, positronemission tomography (PET) systems, PET/MR systems, x-ray systems,ultrasound systems, C-arm systems, and various other imaging modalities.

Medical images from different imaging modalities (e.g., MRI imagesversus CT images) and medical images from the same imaging modality butfrom different imaging systems (e.g., MRI images from two different MRIsystems produced by two different manufactures, or two different modelsof MRI systems produced by a same manufacturer), may possess distinctappearance characteristics attributed to acquisition parameters andmanufacturer-specific image-processing algorithms. In screening anddiagnostic image review, clinicians often need to adapt to modality andmanufacturer-specific image appearance characteristics, as medicalimages for a patient may be acquired using multiple imaging modalitiesor using multiple imaging systems of different models or from differentmanufacturers. Clinicians may have appearance preferences for medicalimages, for example, a clinician may have a greater degree of experiencediagnosing medical images in a first style (that is, medical images witha first set of appearance characteristics), and may therefore prefer toevaluate/diagnose medical images in the first style. Presentation ofmedical images from different imaging modalities/manufacturers/models,which comprise appearance characteristics which do not match with aclinician's preferences, may require additional efforts for a clinicianto adopt to the different appearance characteristics, therefore hinder aclinician's review and diagnostic efficiency. Similarly, deep neuralnetworks and other machine learning models trained to evaluate medicalimages of a first style may perform poorly (e.g., with reduced accuracy)when evaluating medical images in styles distinct from the first style(that is, medical images with appearance characteristics different fromthose of the first style).

Some previous approaches to address the above identified issues attemptto match physical parameters of an originating image processing deviceand a target image processing device (the image processing deviceproducing images matching a clinician's appearance characteristicpreferences), requiring knowledge of the physical parameters andsettings of both systems, which may, in some examples, requirelaboratory testing of both the originating medical imaging device andthe target medical imaging device. Acquisition of the physicalparameters of image processing devices may require substantial time, andmay need to be repeated for each distinct imaging system and imagingmodality. Thus, the above approach may not scale efficiently to largernumbers of image processing devices, and may be impractical in terms ofimplementation time and expense. Other approaches based on conventionalimage feature analysis or supervised machine learning may requiredatasets comprising extensive, manually selected corresponding pairs ofimages of the same anatomical region (e.g., a first medical image of theanatomical region in an originating style, and a second medical image ofthe anatomical region in a target style). However, the acquisition ofsuch pair datasets is often challenging in practice. Additionally,current attempts to map medical images from an originating style to atarget style do not explicitly control the clinical image quality duringthe mapping process, thus may incur substantial clinical qualitydegradation (e.g., a morphology of a tumor is altered upon adjustingimage appearance characteristics to match a designated target style,thereby reducing a clinician's ability to detect the presence of thetumor in the style transferred image).

Therefore, based on the above issues and limitations, it is generallydesirable to explore techniques for transferring style of medicalimages, without requiring knowledge of the physical parameters of theoriginating or target image processing devices, without requiring manualselections of paired images with the same anatomical region, whilepreserving the clinical quality of the medical images.

SUMMARY

The current disclosure at least partially addresses one or more of theabove identified issues. In one embodiment, the current disclosureprovides a method for transferring style of a medical image to a targetstyle, while maintaining the anatomical content of the medical image, aswell as the clinical/diagnostic quality of the medical image,comprising, acquiring a medical image of an anatomical region of asubject, wherein the medical image is in a first style, selecting atarget style, wherein the target style is distinct from the first style,selecting a clinical quality metric, selecting a trained style transfernetwork based on the target style and the clinical quality metric,mapping the medical image to a style transferred medical image using thetrained style transfer network, wherein the style transferred medicalimage is in the target style, and displaying the style transferredmedical image via a display device. The first style and the target styleof the medical image are implicitly characterized by the correspondingun-paired datasets of acquired medical images with the first style andacquired medical images with the target style and are automaticallylearned by the style transfer network during an unsupervised trainingprocess. In this way, one or more appearance characteristics of amedical image may be mapped to one or more target appearancecharacteristics (also referred to herein as a target style), withneither measurement of the physical parameters of a target ororiginating image processing device, nor manual selection of pairedimages. Further, by selecting a trained style transfer network based ona clinical quality metric, the clinical quality of the medical imagewith respect to one or more clinical qualities being evaluated by aclinician or downstream image processing system, may be better preservedin the style transferred medical image.

The above advantages and other advantages, and features of the presentdescription will be readily apparent from the following DetailedDescription when taken alone or in connection with the accompanyingdrawings. It should be understood that the summary above is provided tointroduce in simplified form a selection of concepts that are furtherdescribed in the detailed description. It is not meant to identify keyor essential features of the claimed subject matter, the scope of whichis defined uniquely by the claims that follow the detailed description.Furthermore, the claimed subject matter is not limited toimplementations that solve any disadvantages noted above or in any partof this disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

Various aspects of this disclosure may be better understood upon readingthe following detailed description and upon reference to the drawings inwhich:

FIG. 1 shows a block diagram of an exemplary embodiment of a styletransfer network training system;

FIG. 2 shows a block diagram of an exemplary embodiment of an imageprocessing system configured to map medical images from an originatingstyle to a target style, using a trained style transfer network;

FIG. 3 shows a flowchart of an exemplary method for mapping one or moremedical images from a first style to a target style, using a trainedstyle transfer network;

FIG. 4 shows a flowchart of an exemplary method for training a styletransfer network using the style transfer network training system ofFIG. 1 ; and

FIG. 5 shows an example of a first medical image and a correspondingstyle transferred medical image produced by a trained style transfernetwork.

The drawings illustrate specific aspects of the described systems andmethods for mapping one or more medical images in a first style to oneor more corresponding medical images in a target style using deep neuralnetworks. Together with the following description, the drawingsdemonstrate and explain the structures, methods, and principlesdescribed herein. In the drawings, the size of components may beexaggerated or otherwise modified for clarity. Well-known structures,materials, or operations are not shown or described in detail to avoidobscuring aspects of the described components, systems and methods.

DETAILED DESCRIPTION

The following description relates to systems and methods for mappingmedical images from a first style, to a target style, while preservingclinical/diagnostic quality of the medical images, using one or moredeep neural networks. As an example, breast images from differentimaging modalities and manufacturers present their unique appearancecharacteristics attributed to acquisition specification andmanufacturer-specific image-processing algorithms. In screening anddiagnostic image review, clinicians often need to adapt to modality- andmanufacturer-specific image appearance characteristics for optimizedreview efficiency. Potentially, clinicians' efficiency can be hinderedwhen modality and/or manufacturer changes, or when available reviewsettings cannot meet clinicians' preference for the optimal review.Thus, to increase clinical review efficiency, an appearance transferalgorithm can be used, offering a transition between original imageappearance and a target image appearance that the clinicians are adaptedto. Existing transfer algorithms are either based on matching physicalparameters of original and target imaging systems, requiring priorknowledge of physical parameters and laboratory measurements on bothsystems, or based on matching image-to-image characteristics, requiringselection of a specific target image to pair-up with the original image.Also, quantitative metrics are currently unavailable to increase aprobability that the style transfer does not hinder task-based clinicalimage quality, such as lesion detectability.

Further, to avoid manual selection of target images and physicalmeasurements on imaging systems, which may be prohibitively timeconsuming and/or expensive, an appearance transfer algorithm directlytrainable from unpaired and unlabeled images is desired. Additionally,it may be desirable to incorporate constraints on task-based clinicalimage quality of transferred images, to reduce a probability of clinicalquality degradation from the style transfer process.

The current disclosure provides systems and methods for transformationof an original image, having an original appearance, to a styletransferred medical image, having target set of appearancecharacteristics (wherein the target set of appearance characteristicsmay herein be referred to as a target style). In one embodiment, theoriginal image may be transformed to the target style using a trainedstyle transfer network. The current disclosure further provides fortraining systems and methods, enabling a style transfer network to betrained in an unsupervised fashion to learn a mapping from a first styleto a target style. The training systems and methods disclosed herein donot require measurement of physical characteristics of image processingdevices, nor do the training systems and methods require manuallylabeled images, or pairs of images. Further, the training systems andmethods disclosed herein include constraints on clinical quality, whichmay reduce a probability of clinical quality degradation in styletransferred medical images.

In one embodiment, an image processing device, such as image processingdevice 202, shown in FIG. 2 , may train a style transfer network 106using a style transfer network training system 100, shown in FIG. 1 byexecuting one or more operations of method 400, shown in FIG. 4 . Method400 comprises training the style transfer network 106 to learn a mappingfrom a first style domain to a target style domain, by adjustingparameters to reduce a cumulative loss. The cumulative loss comprises asimilarity loss 114, which may be determined based on output from thestyle similarity estimator 110, a content loss 122, which may bedetermined by an image content regularizer 120, and a clinical qualityloss 116, which may be determined based on output from a clinicalquality estimator 112. The clinical quality loss 116 helps ensure one ormore aspects of clinical quality is maintained through the mapping fromthe first style to the target style by imposing a penalty on the styletransfer network 106 based on changes in clinical quality between inputmedical images 102 in the first style, and the corresponding styletransferred medical images 108 in the target style.

Trained style transfer networks may be implemented by an imageprocessing device, such as image processing device 202, to map one ormore medical images from the first style to one or more correspondingstyle transferred medical images, in the target style, by executing oneor more operations of method 300, shown in FIG. 3 . One example of aninput medical image in a first style, and a corresponding styletransferred medical image, in a target style, which may be produced by atrained style transfer network, is shown in FIG. 5 .

Referring to FIG. 1 , an example of a style transfer network trainingsystem 100 is shown. Style transfer network training system 100 may beimplemented by one or more computing systems, such as image processingdevice 202, shown in FIG. 2 , to train a style transfer network to learna mapping from a first style domain to a target style domain. Styletransfer network training system 100 comprises a style transfer network106 (to be trained), a style similarity estimator 110, a clinicalquality estimator 112, and an image content regularizer 120. The stylesimilarity estimator 110, the clinical quality estimator 112, and theimage content regularizer 120, determine a similarity loss 114, aclinical quality loss 116, and a content loss 122, respectively, whichare aggregated to form a cumulative loss 118. The parameters of styletransfer network 106 are then iteratively updated based on thecumulative loss 118 during the training process.

Style transfer network training system 100 receives as input one or moretarget images 104, and one or more medical images 102, wherein thetarget images 104 differ in at least one appearance characteristic withrespect to medical images 102. In some embodiments, appearancecharacteristics (herein also referred to as visual characteristics)comprise features such as, but not limited to brightness, color,contrast, shading, texture, sharpness and noise level etc.

The number of target images 104 and the number of medical images 102 maybe different or equal, but need not necessarily belong to a samepatient, or include a same anatomical region. Further, the target images104 and the medical images 102 are not paired or labeled, such as inconventional supervised training. The target images 104 and medicalimages 102 may comprise medical images from different patients, maycomprise different anatomical regions, may be different in number, andmay be unlabeled. In particular, the appearance characteristics oftarget images 104 and medical images 102 need not be explicitlyidentified or labeled, as the style similarity estimator 110 may beconfigured to automatically identify and extract appearancecharacteristics from input images. By bypassing the need for labeledtraining data, style transfer network training system 100 greatlyincreases the scalability and efficiency of training style transfernetworks for a plurality of different style transfer mappings, as theamount of usable data is greatly increased, while the amount ofpre-processing of the training data is greatly reduced.

In some embodiments, medical images 102 may be from a first imagingmodality and/or a first imaging system, while target images 104 may befrom a second imaging modality and/or a second imaging system. In someembodiments, target images 104 may comprise medical images having one ormore manually adjusted appearance characteristics, wherein the manuallyadjusted appearance characteristics may be set by a clinician, to suitthe clinician's preferences. As an example, a clinician may preferparticular brightness and contrast settings, and may manually adjust thebrightness and contrast of one or more medical images to meet thesepreferences. These manually brightness and contrast adjusted images maybe fed to style transfer training system 100 as target images 104, and astyle transfer network 106 may be trained to automatically mapappearance characteristics of acquired medical images to the clinicianpreferred appearance characteristics of target images 104.

Style transfer network training system 100 may be implemented accordingto one or more operations of method 400, to train a style transfernetwork to learn a mapping from the style domain characterized bymedical images 102, to the style domain characterized by target images104. Style transfer network 106 is configured to receive data frommedical images 102, and to map this data to corresponding styletransferred medical images 108. In some embodiments, style transfernetwork 106 comprises a parametric generative transfer model. In someembodiments, style transfer network 106 may comprise a deep neuralnetwork. In some embodiments, style transfer network 106 may comprise adeep neural network having a U-net architecture. In some embodiments,style transfer network 106 may comprise a deep neural network having avariational autoencoder architecture, comprising a first encodingportion, which compresses the information of medical images 102 into acondensed representation/encoding, and a decoder portion, whichdecompresses the condensed representation/encoding to a variation of themedical images 102. In some embodiments, the encoding portion comprisesone or more convolutional layers, which in turn comprise one or moreconvolutional filters. The convolutional filters may comprise aplurality of weights, wherein the values of the weights are learnedduring a training procedure, such as the training method of FIG. 4 . Theconvolutional filters may correspond to one or more visualfeatures/patterns, thereby enabling the style transfer network 106 toidentify and extract visual features from input medical images 102. Theencoding portion may further comprise one or more down samplingoperations, and/or one or more activation functions. The decodingportion may comprise one or more up-sampling, and/or deconvolutionoperations, which enable a compressed representation of the medicalimages 102 to be reconstructed into an image of the same size as theinput medical images 102.

Output of style transfer network 106 may be used to generate styletransferred medical images 108. In some embodiments, style transfernetwork 106 may directly output style transferred medical images 108.Style transferred medical images 108 may comprise a same number ofimages as medical images 102, wherein for each image of medical images102, a corresponding style transferred medical image is produced, suchthat there is a 1-to-1 correspondence between medical images 102 andstyle transferred medical images 108. Style transferred medical images108, along with medical images 102 and/or target images 104 may be fedto style similarity estimator 110, clinical quality estimator 112, andimage content regularizer 120, to determine a similarity loss 114, aclinical quality loss 116, and a content loss 122, respectively. Thevarious losses may be aggregated to form a cumulative loss 118, whereinthe cumulative loss 118 may be used to update parameters of styletransfer network 106, such as is described in method 400 of FIG. 4 .

The style transferred medical images 108, along with target images 104,may be fed to style similarity estimator 110, for determination of asimilarity loss 114. The similarity loss 114 represents a quantificationof the differences in appearance between style transferred medicalimages 108 and target images 104. In other words, style similarityestimator 110 evaluates the visual characteristics of style transferredmedical images 108, and compares this with the visual characteristics oftarget images 104, to produce a numerical value indicating a degree ofstylistic similarity/difference between target images 104 and styletransferred medical images 108.

Style similarity estimator 110 may comprise one or more differentiablefunctions, configured to receive style transferred medical images 108and target images 104 as input, and output a similarity score/similarityloss 114, indicating the similarity in appearance between the styletransferred medical images 108 and the target images 104. In someembodiments, style similarity estimator 110 is configured to output aprobability of an input image belonging to the set of target images 104.In one embodiment, style similarity estimator 110 may comprise adiscriminator network. The discriminator network may be trained tooutput the probability that an input image belongs to the target images104. Similarity loss 114 may increase in response to increased accuracyof the discriminator network correctly distinguishing between targetimages 104 and style transferred medical images 108. Similarity loss 114may decrease in response to decreased accuracy (e.g., an accuracy of 0.5indicates the accuracy of a random guess, whereas an accuracy of 1.0indicates the discriminator is able to distinguish target images fromstyle transferred images with 100% accuracy) of the discriminatornetwork correctly distinguishing between target images 104 and styletransferred medical images 108. The parameters of style transfer network106 may be adjusted to reduce the similarity loss 114, which has theeffect of increasing the visual similarity of style transferred medicalimages 108 to target images 104.

Similarly, the clinical quality estimator 112 may receive both medicalimages 102, and style transferred medical images 108, and determine aclinical quality loss 116 based thereon. The clinical quality loss 116represents a difference in clinical quality for one or more clinicalquality metrics, between medical images 102 and style transferredmedical images 108. In some embodiments, clinical quality estimator 112may comprise a deep convolutional neural network, trained to identifyone or more clinically relevant features in an input image, and toproduce a score indicating one or more attributes of the identifiedclinically relevant feature. In some embodiments, clinical qualityestimator 112 comprises a deep neural network pre-trained to produce acancer classification score for input medical images, and clinicalquality loss 116 may increase as the difference in cancer score betweenmedical images 102 and style transferred medical images 108 increases.In some embodiments, clinical quality estimator 112 comprises a modelobserver producing a lesion detectability index for input medicalimages, and clinical quality loss 116 may increase as the difference inlesion detectability between medical images 102 and style transferredmedical images 108 increases. In other words, clinical quality estimator112 forces style transfer network 106 to maintain one or more clinicalquality metrics of style transferred medical images 108 within athreshold difference of the clinical quality metrics of medical images102.

Style transfer network training system 100 further comprises imagecontent regularizer 120. Image content regularizer may comprise one ormore differentiable functions, which take medical images 102 and styletransferred medical images 108 as input, and produce content loss 122 asoutput. Image content regularizer 120 may be configured such that as thedifference in anatomical content between an input medical image, and anoutput style transferred image increases, the content loss 122 alsoincreases. Image content regularizer may be configured to computepixelwise distances between medical images 102 and style transferredmedical images 108, wherein content loss 122 may be based on the summedor averaged pixelwise distances. The distance metric used in the imagecontent regularizer may be, but are not limited to the L1 or the L2distance. Image content regularizer 120 helps tether the overallcontent, as opposed to the style, of style transferred medical images108, to the content of medical images 102. In other words, byintroducing a content loss 122, produced by image content regularizer120, into the cumulative loss 118, and by training style transfernetwork 106 to minimize the cumulative loss 118, style transfer network106 may produce style transferred medical images 108 which preserve theanatomical content of medical images 102.

The similarity loss 114, the content loss 122, and the clinical qualityloss 116, may be aggregated to form a cumulative loss 118, wherein thecumulative loss 118 may comprise a weighted sum or weighted average ofthe similarity loss 114, the content loss 122, and the clinical qualityloss 116. In some embodiments, each of the similarity loss 114, thecontent loss 122, and the clinical quality loss 116, may have anassociated weight (that is, three distinct weights may be used todetermine the cumulative loss 118, one for each of the above said lossterms), and the cumulative loss 118 may be determined by multiplyingeach of similarity loss 114, the content loss 122, and the clinicalquality loss 116 by its associated weight, and summing the products soobtained.

The weighting of each of the loss terms during determination of thecumulative loss 118 serves to provide flexibility in the relativeimportance of each type of loss in the training process of styletransfer network 106, thereby enabling style transfer network 106 tolearn a user customizable style transfer mapping, wherein a relativeimportance of anatomical content conservation, clinical qualityconservation, and target style similarity, are adjustable for training.As an example, setting a weight given to similarity loss 114 to asignificantly larger value than the weights for content loss 122, andclinical quality loss 116, may cause style transfer network 106 to learna mapping which favors similarity between the target images 104 and thestyle transferred images 108, at the expense of clinical qualityconservation and content conservation. Conversely, giving a relativelylarge weight to clinical quality loss ensures that clinical quality ismaintained between the medical images 102 and the style transferredimages 108, whereas the visual similarity between the style transferredimages 108 and the target images 104 may increase. In some embodiments,the weights used to determine cumulative loss 118 may be manually set.In some embodiments, the weights used to determine cumulative loss 118may be learned via automatic hyper-parameter search. In one embodiment,the weights used to determine the cumulative loss 118 may be determinedusing Bayesian optimization or reinforcement learning.

In this way, style transfer network training system 100 enables a styletransfer network 106 to learn a map from a first style domain, to atarget style domain, while reducing changes in clinical quality oroverall anatomical content which may otherwise result from the mapping.Further, by utilizing the style similarity network 110 to automaticallyidentify visual features from both target images 104, and styletransferred medical images 108, a similarity loss reflective of ahumanly intuitive visual similarity may be obtained, without reliance onuse of manually labeled training data sets.

Referring to FIG. 2 , a medical imaging system 200 is shown, inaccordance with an exemplary embodiment. Medical imaging system 200comprises image processing device 202, display device 220, user inputdevice 230, and medical imaging device 240. In some embodiments, atleast a portion of medical imaging system 200 is disposed at a remotedevice (e.g., edge device, server, etc.) communicably coupled to themedical imaging system 200 via wired and/or wireless connections. Insome embodiments, at least a portion of image processing device 202 isdisposed at a separate device (e.g., a workstation) configured toreceive images from a storage device which stores images acquired bymedical imaging device 240.

Image processing device 202 includes a processor 204 configured toexecute machine readable instructions stored in non-transitory memory206. Processor 204 may be single core or multi-core, and the programsexecuted thereon may be configured for parallel or distributedprocessing. In some embodiments, the processor 204 may optionallyinclude individual components that are distributed throughout two ormore devices, which may be remotely located and/or configured forcoordinated processing. In some embodiments, one or more aspects of theprocessor 204 may be virtualized and executed by remotely-accessiblenetworked computing devices configured in a cloud computingconfiguration.

Non-transitory memory 206 may store deep neural network module 208,training module 212, and image data 214. Deep neural network module 208may include one or more deep neural networks, comprising a plurality ofweights and biases, activation functions, loss functions, andinstructions for implementing the one or more deep neural networks tomap medical images of a first style to corresponding medical images in atarget style. For example, deep neural network module 208 may storeinstructions for implementing one or more neural networks, according toone more steps of method 300, discussed in more detail below. Deepneural network module 208 may include trained and/or untrained neuralnetworks. In some embodiments, the deep neural network module 208 is notdisposed at the image processing device 202, but is disposed at a remotedevice communicably coupled with image processing device 202 via wiredor wireless connection. Deep neural network module 208 may includevarious deep neural network metadata pertaining to the trained and/oruntrained networks. In some embodiments, the deep neural networkmetadata may include an indication of the training data used to train adeep neural network, a training method employed to train a deep neuralnetwork, an accuracy/validation score of a deep neural network, and atype clinical quality metric and target style for which the trained deepneural network may be applied.

Non-transitory memory 206 further includes training module 212, whichcomprises machine executable instructions for training one or more ofthe deep neural networks stored in deep neural network module 208. Insome embodiments, training module 212 may include instructions forimplementing a style transfer network training system, such as styletransfer network training system 100, shown in FIG. 1 . In oneembodiment, the training module 212 may include gradient descentalgorithms, loss functions, and rules for generating and/or selectingtraining data for use in training a particular deep neural network.Training module 212 may further include instructions, that when executedby processor 204, cause image processing device 202 to train a styletransfer network by executing one or more of the operations of method400, using target images selected based on a user selected target style,and using a clinical quality estimator selected based on one or moreuser selected clinical quality metrics, as discussed in more detail withreference to FIG. 4 , below. In some embodiments, the training module212 is not disposed at the image processing device 202, but is disposedremotely, and is communicably coupled with image processing device 202.

Non-transitory memory 206 may further store image data 214, comprisingmedical imaging data acquired by medical imaging device 240. The medicalimages stored in image data 214 may comprise medical images from variousimaging modalities and/or from various models/manufacturers of medicalimaging devices. In some embodiments, medical images stored in imagedata 214 may include information identifying an imaging modality and/oran imaging device (e.g., model and manufacturer of an imaging device) bywhich the medical image was acquired. In some embodiments, medicalimages stored in image data 214 may include regions manually highlightedby clinicians using the user input device 230, wherein the anatomicalcontent of the highlighted region may be prioritized for preservationduring the style transfer. In some embodiments, highlighted regions maycomprise regions suspected by a clinician to include a lesion/tumor orother pathology. These highlighted regions can be explored by theclinical image quality estimator during training. In some embodiments,image data 214 may comprise MR images captured by an MRI system, CTimages captured by a CT imaging system, PET images captured by a PETsystem, breast images captured by a Mammography system and/or one ormore additional types of medical images. In one embodiment, image data214 may comprise representative target images from a plurality ofdistinct imaging styles (e.g., different imaging modalities, differentimage processing algorithms, etc.).

In some embodiments, the non-transitory memory 206 may includecomponents disposed at two or more devices, which may be remotelylocated and/or configured for coordinated processing. In someembodiments, one or more aspects of the non-transitory memory 206 mayinclude remotely-accessible networked storage devices configured in acloud computing configuration.

Medical imaging system 200 further includes imaging device 240, whichmay comprise substantially any type of medical imaging device, includingMRI, CT, PET, hybrid PET/MR, Mammography, ultrasound, etc. Imagingdevice 240 may acquire measurement data of an anatomical region of apatient, which may be used to generate medical images. Measurement data,and medical images reconstructed therefrom, may be stored in image data214, or in other non-transitory storage devices communicably coupledwith image processing device 202.

Medical imaging system 200 may further include user input device 230.User input device 230 may comprise one or more of a touchscreen, akeyboard, a mouse, a trackpad, a motion sensing camera, or other deviceconfigured to enable a user to interact with and manipulate data withinimage processing device 202.

Display device 220 may include one or more display devices utilizingvirtually any type of technology. In some embodiments, display device220 may comprise a computer monitor configured to display medical imagesof various types and styles. Display device 220 may be combined withprocessor 204, non-transitory memory 206, and/or user input device 230in a shared enclosure, or may be a peripheral display device and maycomprise a monitor, touchscreen, projector, or other display deviceknown in the art, which may enable a user to view medical imagesreconstructed from measurement data acquired by imaging device 240,and/or interact with various data stored in non-transitory memory 206.

It should be understood that medical imaging system 200 shown in FIG. 2is for illustration, not for limitation. Another appropriate medicalimaging system 200 may include more, fewer, or different components.

Turning to FIG. 3 , an example method 300 for mapping medical imageshaving a first set of appearance characteristics (a first style) toanatomically consistent medical images with a different, target set ofappearance characteristics (a target style) is shown. Method 300 may beexecuted by one or more of the systems described above. In oneembodiment, image processing device 202 may execute one or moreoperations of method 300 to map a medical image acquired by imagingdevice 240 to a clinician selected target style. In this way, aclinician may evaluate/diagnose medical images with a preferred set ofappearance characteristics.

Method 300 begins at operation 302, where the medical imaging systemacquires a medical image of an anatomical region of a patient. In someembodiments, operation 302 may include medical imaging device 240acquiring one or more medical images of an anatomical region of interestfrom a patient. In one example, operation 302 may comprise a medicalimaging device acquiring a mammogram image of a patient. Medical imagesmay be acquired using one or more known imaging modalities, includingMRI, PET, CT, x-ray, ultrasound, etc. Acquired medical images maycomprise two-dimensional (2D) or three-dimensional (3D) images. Theacquired medical images may comprise a first set of appearancecharacteristics, indicative of the imaging modality and/or imagereconstruction algorithm and/or image processing algorithms used by themedical imaging device during acquisition and reconstruction/processingof said medical images. The set of appearance characteristics of theacquired medical images may be referred to as a first style, ororiginating style. In some embodiments, acquired medical images may betransmitted to a remote image processing device. In some embodiments,the medical imaging system includes a built in image processing device.

At operation 304, the image processing device receives a target styleselection. The target style selection indicates a preferred set ofappearance characteristics. One or more of the appearancecharacteristics of the preferred set of appearance characteristic maydiffer from one or more of the appearance characteristics of theacquired medical image. In other words, a style defined by the targetstyle selection may differ from the first style. In some embodiments,operation 304 includes the medical imaging system receiving a targetstyle selection from a user via a user input device. In someembodiments, the image processing device may obtain/retrieve apre-selected target style selection stored in a location ofnon-transitory memory wherein a user's preferences are stored. In someembodiments, a target style selection may comprise a designation of aparticular imaging modality, which may differ from the imaging modalityused to acquire the medical images at operation 302. In someembodiments, the target style selection may comprise a selection of aset of appearance characteristics corresponding to a particularmanufacturer or model of an imaging device, wherein the medical imagingsystem of operation 302 may differ from the particular manufacturer ormodel of the imaging device of the target style. In some embodiments,the target style selection may comprise a user customized style,comprising one or more user selected appearance characteristics. As anexample, a clinician may manually adjust a set of parameters related toimage appearance characteristics, including but not limited to contrast,sharpness or other high-level semantic appearance parameters extractedfrom a neural network such as an autoencoder etc., and may set thesepreferences as a target style.

At operation 306, the image processing device receives a clinicalquality metric selection. In some embodiments, the medical imagingsystem may receive the clinical quality metric selection from a user viaa user input device. In some embodiments, the imaging system may obtaina pre-selected clinical quality metric from a location in non-transitorymemory, wherein user preferences are stored. The selected clinicalquality metric selection may comprise one or more of lesiondetectability, bone fracture detectability, vasculature visualization,etc. In case where clinician highlighted cancer regions, as described in[0037] are available, the clinical quality metric may also comprise acancer classification score produced by a pre-trained cancer classifieron the highlighted regions. A clinician may select a clinical qualitymetric relevant to a current task, for example, during screening ofmedical images for the presence of tumors, a clinician may select (ormay have pre-selected) a tumor detectability index/score as the clinicalquality metric. In some embodiments, more than one clinical qualitymetric may be selected at operation 306. In some embodiments, apre-determined number of clinical quality metrics may be selected incombination.

At operation 308, the image processing device selects a trained styletransfer network based on the target style selection and the clinicalquality metric selection. Selecting the trained style transfer networkbased on the selected target style and clinical quality metrics maycomprise selecting a pre-trained style transfer network, trained by astyle transfer network training system, such as style transfer networktraining system 100, according to one or more operations of method 400.In particular, the trained style transfer network selected at operation308 comprises a style transfer network trained using a clinical qualityloss based on the one or more clinical quality metrics selected atoperation 306, and a similarity loss based on a visual differencebetween style transferred medical images produced by the style transfernetwork, and medical images of the target style (wherein the targetstyle is the same target style selected at operation 304). Further, thetrained style transfer network may include one or more pieces ofmetadata, indicating one or more training parameters, such as a clinicalquality estimator used during training, a clinical quality metricevaluated by the clinical quality estimator, a target style domain towhich the style transfer network learned a mapping, and a first styledomain and/or a first style for which the trained style transfer networkhas learned a mapping, etc. Therefore, operation 308 may comprise theimage processing device selecting a trained style transfer network,based on one or more pieces of metadata associated with the styletransfer network indicating the style transfer network was trained usinga clinical quality loss and style loss matching the clinical qualitymetric selection and the target style selection.

At operation 310, the image processing device maps the medical image toa style transferred medical image using the trained style transfernetwork. Operation 310 may include inputting data from the medical imageinto the trained style transfer network. The trained style transfernetwork may automatically identify composing appearance characteristicsof the input medical image data, and may project the identifiedappearance characteristics to the target style domain. As a non-limitingexample, the trained style transfer network may receive a medical imagehaving features of a first shape, brightness, color, and texture, andthe style transfer network may identify the features and map theidentified features to corresponding features in a target style domain,wherein, as an example, the corresponding features in the target styledomain may comprise a similar or the same shape, a different brightness,a different texture, and a different color. In some embodiments, thetrained style transfer network may comprise a deep neural network, andoperation 310 may include inputting the one or more medical images inthe first style into an input layer of the trained deep neural network.The trained deep neural network may comprise one or more convolutionallayers, comprising learned filters, which may identify appearancecharacteristics/features of the input medical image.

Turning briefly to FIG. 5 , one example of an input medical image 502,in a first style, being mapped to a style transferred medical image 504,in a target style, by a trained style transfer network 506, is shown.Input medical image 502 may comprise a medical image acquired by a firstmedical imaging system. Images acquired by the first medical imagingsystem may not meet a clinician's image appearance preferences, and inresponse, the clinician may employ a method, such as method 300, to mapthe input medical image 502 to a style transferred medical image 504,wherein the style transferred medical image 504 comprises the imageappearance preferences of the clinician (that is, the appearancecharacteristics of the style transferred medical image 504 match atarget style defining the preferred image appearance characteristics ofthe clinician). Both input medical image 502 and style transferredmedical image 504 comprise substantially the same anatomical content,and are of the same anatomical region of a same patient, however, theinput medical image 502 and the style transferred medical image 504comprise different appearance characteristics (that is, differentstyle). Trained style transfer network 506, may have been selected basedon a clinician's target style selection and clinical quality metricselection, as described above with reference to operation 304 andoperation 306. Trained style transfer network 506 comprises a learnedmap from the first style domain (that is, the set of appearancecharacteristics of the input medical image 502) to a target style domain(that is, the set of appearance characteristics of the style transferredmedical image 504).

Returning to FIG. 3 , at operation 312, the image processing deviceoptionally determines a clinical quality score for the style transferredmedical image. In some embodiments, determining a clinical quality scorefor the style transferred medical images may comprise inputting both thestyle transferred medical image, and the input medical image, into aclinical quality estimator, wherein the clinical quality estimatorcomprises one or more differentiable functions, and wherein the clinicalquality estimator evaluates a difference in the selected clinicalquality metric between the input medical image and the style transferredmedical image. Therefore, the clinical quality score indicates adifference in clinical quality between the input medical image, and thecorresponding style transferred medical image.

At operation 314, the medical imaging system optionally determines asimilarity score for the style transferred medical images. In someembodiments, operation 314 comprises selecting a similarity estimatorbased on the target style selection of operation 304, and determining adegree of similarity between the style transferred medical image(s)produced at operation 310, and images of the target style. In someembodiments, the style similarity estimator may comprise a discriminatornetwork, trained to determine a probability of an input image belongingto a selected class (wherein the selected class comprises images of thetarget style). The style transferred medical image may be input into thediscriminator network, and a probability of the style transferredmedical image belonging to the selected class may be output. In someembodiments, the output probability may be used as the similarity score.

At operation 316, the medical imaging system displays the styletransferred medical image via a display device. In some embodiments,operation 316 may comprise displaying the style transferred medicalimage produced at operation 310, via a display device, such as displaydevice 220. In some embodiments, method 300 may optionally includeoperation 318, wherein the medical imaging system displays thesimilarity score and/or the clinical quality score via the displaydevice. In some embodiments, the clinical quality score and/or thesimilarity score may be stored as meta data associated with the styletransferred medical image. In one embodiment, the clinical quality scoreand or the similarity score may be stored in a DICOM header of the styletransferred medical image.

Following operation 318, method 300 may end. Method 300 may enable aclinician to select a preferred target style, and one or more clinicalquality metrics pertaining to a current clinical task (e.g., detectionof lesions, evaluation of vasculature, detection of bone fractures,etc.), and a trained style transfer network may be selected based on theselected target style and clinical quality metrics. The trained styletransfer network may comprise a learned mapping between appearancecharacteristics of the first style domain and corresponding appearancecharacteristics of the target style domain. The mapping may further havebeen learned under a constraint to maintain the selected clinicalquality metric of the input medical images, and output style transferredmedical images within a pre-determined threshold of each other. This mayenable a clinician to evaluate/diagnose medical images in a preferredstyle, without the need for the clinician to manually edit the medicalimages.

Further, by determining clinical quality scores and similarity scoresfor the style transferred medical images, and by displaying the clinicalquality score and/or similarity score (or by making these clinicalquality and similarity scores available for viewing by a clinician), aclinician may be informed regarding changes in clinical quality that mayhave occurred during the mapping from the first style to the targetstyle. Thus, a clinician may make the final determination in decidingwhether or not to use style transferred medical images to perform adiagnosis.

Turning to FIG. 4 , an example of a training method 400, which may beexecuted by one or more of the systems described above, is shown. In oneembodiment, method 400 may be used to train a style transfer network,such as style transfer network 106, shown in FIG. 1 , to map medicalimages from a first style domain to a target style domain, whilepreserving the anatomical content and one or more clinical qualitymetrics of the medical images.

Method 400 begins at operation 402, where the image processing deviceselects a set of medical images in a first style, and a set of targetimages in a target style. The target images differ in at least oneappearance characteristic with respect to the medical images. In someembodiments, appearance characteristics comprise features such asbrightness, color, contrast, shading, texture, etc. The number of targetimages and the number of medical images may be different or equal, butneed not necessarily belong to a same patient, or include a sameanatomical region. Further, the target images and the medical images arenot paired or labeled, such as in conventional supervised training. Thetarget images and medical images may comprise medical images fromdifferent patients, may comprise different anatomical regions, may bedifferent in number, and may be unlabeled.

At operation 404, the image processing device selects a clinical qualityestimator. In some embodiments, the clinical quality estimator maycomprise a deep convolutional neural network, trained to identify one ormore clinically relevant features in an input image, and to produce ascore indicating one or more attributes of the identified clinicallyrelevant feature. In some embodiments, the clinical quality estimatormay comprise a pre-trained deep neural network or a model observer toproduce a lesion detectability score for input medical images.

At operation 406, the image processing device selects a style similarityestimator. The style similarity estimator may comprise one or moredifferentiable functions, configured to receive style transferredmedical images as input, and output a similarity score/similarity loss,indicating the similarity in appearance between the style transferredmedical images and the target images. In some embodiments, the stylesimilarity estimator may comprise a discriminator network. In someembodiments, the style similarity estimator may comprise a siameseneural network. In some embodiments, the style similarity estimator isconfigured to output a probability of an input image belonging to theset of target images.

At operation 407, the image processing device selects an image contentregularizer. The image content regularizer may comprise one or moredifferentiable functions, which take medical images and styletransferred medical images as input, and produce a content loss asoutput. The image content regularizer may be configured such that as thedifference in anatomical content between an input medical image, and anoutput style transferred image increases, the content loss alsoincreases. Image content regularizer may be configured to computepixelwise distances between medical images and style transferred medicalimages, wherein the content loss may be based on the summed or averagedpixelwise distances. The distance metric used in the image contentregularizer may be, but are not limited to the L1 or the L2 distance.The image content regularizer helps tether the overall content, asopposed to the style, of style transferred medical images, to thecontent of medical images.

At operation 408, the image processing device optionally trains thestyle similarity estimator using the set of target images. In someembodiments, operation 408 includes training the similarity estimator toidentify features present in images belonging to the set of targetimages, and to use the identified features to distinguish/discriminatebetween images belonging to the set of target images and imagesbelonging to other sets. In some embodiments, style similarity estimatormay be pre-trained. In some embodiments, style similarity estimator maybe trained at the same time during the training of the style transfernetwork.

At operation 410, the image processing device generates a set of styletransferred medical images from the set of medical images using a styletransfer network. Operation 410 may include inputting data from the setof medical images into the un-trained style transfer network. Theun-trained style transfer network may automatically identify composingappearance characteristics of the input medical image data, and mayproject the identified appearance characteristics to the target styledomain using the initialized/untrained weights. In some embodiments, theun-trained style transfer network may comprise a deep neural network,and operation 410 may include inputting the set of medical images in thefirst style into an input layer of the un-trained deep neural network.

At operation 412, the image processing device determines a clinicalquality loss by comparing the set of style transferred medical images tothe set of medical images using the clinical quality estimator. Theclinical quality loss represents a difference in clinical quality forone or more clinical quality metrics, between the medical images and thecorresponding style transferred medical images. As the difference inclinical quality between an input medical image, and a correspondingstyle transferred medical image produced by the style transfer networkincreases, the clinical quality loss also increases. In one embodiment,the clinical quality loss may comprise an absolute value of a differencein a clinical quality metric determined for an input medical image and asame clinical quality metric determined for a corresponding styletransferred medical image produced by the style transfer network.

At operation 413, the image processing device determines a content lossof the style transferred medical images using an image contentregularizer. The image content regularizer may be configured such thatas the difference in overall anatomical content between an input medicalimage, and an output style transferred image increases, the content lossalso increases. Image content regularizer may be configured to computepixelwise differences between medical images and style transferredmedical images, wherein content loss may be based on the summedpixelwise differences.

At operation 414, the image processing device determines a similarityloss for the style transferred medical images using the style similarityestimator. The similarity loss represents a quantification of thedifferences in appearance between style transferred medical images andtarget images. In other words, the style similarity estimator evaluatesthe visual characteristics of the style transferred medical images, andcompares this with the visual characteristics of the target images, toproduce a numerical value indicating a degree of stylistic similaritybetween the target images and style transferred medical images.

At operation 416, the image processing device aggregates the clinicalquality loss, the content loss, and the similarity loss, to produce acumulative loss. The similarity loss, the content loss, and the clinicalquality loss, may be aggregated to form a cumulative loss, wherein thecumulative loss may comprise a weighted sum or weighted average of thesimilarity loss, the content loss, and the clinical quality loss. Insome embodiments, each of the similarity loss, the content loss, and theclinical quality loss, may have an associated weight (that is, threedistinct weights may be used to determine the cumulative loss, one foreach of the above said loss terms), and the cumulative loss may bedetermined by multiplying each of similarity loss, the content loss, andthe clinical quality loss by its associated weight, and summing theproducts so obtained. In some embodiments, the weights used to determinethe cumulative loss may be manually set. In some embodiments, theweights used to determine the cumulative loss may be learned viaautomatic hyper-parameter search. In one embodiment, the weights used todetermine the cumulative loss may be determined using Bayesianoptimization or reinforcement learning.

At operation 418, the image processing device updates style transfernetwork parameters by minimizing the cumulative loss. In someembodiments, updating the style transfer network parameters includesadjusting parameters of the one or more style transfer networks bybackpropagating the cumulative loss through the layers of the styletransfer network using a backpropagation algorithm. In one embodiment,operation 418 comprises the image processing device adjusting theweights and biases of the layers of the style transfer network based onthe cumulative loss determined at operation 416. In some embodiments,back propagation of the cumulative loss may occur according to agradient descent algorithm, wherein a gradient of the cumulative lossfunction (a first derivative, or approximation of the first derivative)is determined for each weight and bias of the style transfer network.Each weight (and bias) of the style transfer network is then updated byadding the negative of the product of the gradient of the cumulativeloss, determined with respect to the weight (or bias) and apredetermined step size, according to the below equation:

$P_{i + 1} = {P_{i} - {\eta\frac{\partial{Loss}}{\partial P_{i}}}}$where P_(i+1) is the updated parameter value, P_(i) is the previousparameter value, η is the step size, and

$\frac{\partial{Loss}}{\partial P_{i}}$is me partial derivative of me cumulative loss with respect to theprevious parameter.

Following operation 418, method 400 may end. It will be appreciated thatmethod 400 may be repeated until one or more conditions are met. In someembodiments, the one or more conditions may include the weights andbiases of the transfer network converging (that is, a rate of change ofthe parameters decreases to below a pre-determined threshold rate ofchange), the cumulative loss determined at operation 416 decreasing tobelow a pre-determined, non-zero, threshold (in some embodiments, thecumulative loss may be determined using a validation dataset, distinctfrom the training dataset). In this way, method 400 enables a deepneural network to learn a mapping from a first style domain to a targetstyle domain, wherein the mapping preserves the anatomical content and aclinical quality, as determined by the clinical quality estimator, ofthe medical images, thereby preventing degradation of clinical qualityupon mapping of the input medical images to the style transferredmedical images.

When introducing elements of various embodiments of the presentdisclosure, the articles “a,” “an,” and “the” are intended to mean thatthere are one or more of the elements. The terms “first,” “second,” andthe like, do not denote any order, quantity, or importance, but ratherare used to distinguish one element from another. The terms“comprising,” “including,” and “having” are intended to be inclusive andmean that there may be additional elements other than the listedelements. As the terms “connected to,” “coupled to,” etc. are usedherein, one object (e.g., a material, element, structure, member, etc.)can be connected to or coupled to another object regardless of whetherthe one object is directly connected or coupled to the other object orwhether there are one or more intervening objects between the one objectand the other object. In addition, it should be understood thatreferences to “one embodiment” or “an embodiment” of the presentdisclosure are not intended to be interpreted as excluding the existenceof additional embodiments that also incorporate the recited features.

In addition to any previously indicated modification, numerous othervariations and alternative arrangements may be devised by those skilledin the art without departing from the spirit and scope of thisdescription, and appended claims are intended to cover suchmodifications and arrangements. Thus, while the information has beendescribed above with particularity and detail in connection with what ispresently deemed to be the most practical and preferred aspects, it willbe apparent to those of ordinary skill in the art that numerousmodifications, including, but not limited to, form, function, manner ofoperation and use may be made without departing from the principles andconcepts set forth herein. Also, as used herein, the examples andembodiments, in all respects, are meant to be illustrative only andshould not be construed to be limiting in any manner.

The invention claimed is:
 1. A method comprising: selecting a set ofmedical images, wherein the set of medical images are in a first style;selecting a set of target images, wherein the set of target images arein a second style, and wherein the set of medical images and the set oftarget images are unpaired; selecting a clinical quality estimator;selecting a style similarity estimator; selecting an image contentregularizer; training a style transfer network using the set of medicalimages, the set of target images, the clinical quality estimator, thestyle similarity estimator, and the image content regularizer, toproduce a trained style transfer network; receiving a medical image,wherein the medical image is in the first style; mapping the medicalimage to a style transferred medical image using the trained styletransfer network, wherein the style transferred medical image is in thesecond style; displaying the style transferred medical image via adisplay device; and wherein training the style transfer network usingthe set of medical images, the set of target images, the clinicalquality estimator, the style similarity estimator, and the image contentregularizer, to produce the trained style transfer network comprisestraining the style similarity estimator using the set of target images,and wherein the style similarity estimator comprises a neural network,and wherein training the style similarity estimator using the set oftarget images comprises: selecting at random an image from either theset of target images or a set of non-target images; mapping the image toa probability score using the style similarity estimator, wherein theprobability score indicates a probability of the image belonging to theset of target images; calculating a loss based on the probability score;and updating parameters of the style similarity estimator to reduce theloss using a gradient descent algorithm.
 2. The method of claim 1,wherein training the style transfer network using the set of medicalimages, the set of target images, the clinical quality estimator, thestyle similarity estimator, and the image content regularizer, toproduce the trained style transfer network comprises: generating a setof style transferred medical images from the set of medical images usingthe style transfer network; determining a clinical quality loss bycomparing the set of style transferred medical images to the set ofmedical images using the clinical quality estimator; determining asimilarity loss for the set of style transferred medical images usingthe style similarity estimator; determining a content loss by comparingthe set of style transferred medical images to the set of medical imagesusing the image content regularizer; aggregating the clinical qualityloss, the similarity loss and the content loss to produce a cumulativeloss; and adjusting one or more parameters of the style transfer networkbased on the cumulative loss.
 3. The method of claim 2, whereinadjusting one or more parameters of the style transfer network based onthe cumulative loss comprises: backpropagating the cumulative lossthrough the clinical quality estimator, the style similarity estimator,the image content regularizer, and the style transfer network, toproduce a backpropagated cumulative loss; and updating the one or moreparameters of the style transfer network based on the backpropagatedcumulative loss using a gradient descent algorithm.
 4. The method ofclaim 2, the method further comprising: responding to the cumulativeloss decreasing to below a cumulative loss threshold by: storing thestyle transfer network in a location of non-transitory memory, alongwith one or more pieces of meta data identifying the clinical qualityestimator, the style similarity estimator, and the image contentregularizer.
 5. The method of claim 2, wherein aggregating the clinicalquality loss, the similarity loss, and the content loss to produce thecumulative loss comprises: multiplying the clinical quality loss by afirst weight to produce a weighted clinical quality loss; multiplyingthe similarity loss by a second weight to produce a weighted similarityloss; multiplying the content loss by a third weight to produce aweighted image content loss; and summing the weighted clinical qualityloss, the weighted similarity loss, and the weighted content loss toproduce the cumulative loss.
 6. The method of claim 2, wherein theclinical quality estimator comprises a pre-trained neural network, andwherein determining the clinical quality loss by comparing the set ofstyle transferred medical images to the set of medical images using theclinical quality estimator comprises: mapping the set of styletransferred medical images to a first plurality of clinical qualityscores using the clinical quality estimator; mapping the set of medicalimages to a second plurality of clinical quality scores using theclinical quality estimator; determining pair-wise differences betweenthe first plurality of clinical quality scores and the second pluralityof clinical quality scores to produce a plurality of clinical qualitylosses; and summing an absolute value of each of the plurality ofclinical quality losses to produce the clinical quality loss.
 7. Animage processing system comprising: a display device; a memory storing atrained style transfer network and instructions; and a processorcommunicably coupled to the display device and the memory, and whenexecuting the instructions, configured to: select a set of medicalimages, wherein the set of medical images are in a first style; select aset of target images, wherein the set of target images are in a secondstyle, and wherein the set of medical images and the set of targetimages are unpaired; select a clinical quality estimator; select a stylesimilarity estimator; select an image content regularizer; train a styletransfer network using the set of medical images, the set of targetimages, the clinical quality estimator, the style similarity estimator,and the image content regularizer, to produce a trained style transfernetwork; receive a medical image, wherein the medical image is in thefirst style; map the medical image to a style transferred medical imageusing the trained style transfer network, wherein the style transferredmedical image is in the second style; display the style transferredmedical image via the display device; and wherein the processor, whenexecuting the instructions, to train the style transfer network usingthe set of medical images, the set of target images, the clinicalquality estimator, the style similarity estimator, and the image contentregularizer, to produce the trained style transfer network is configuredto train the style similarity estimator using the set of target images;and wherein the style similarity estimator comprises a neural network,and wherein the processor, when executing the instructions, to train thestyle similarity estimator using the set of target images is configuredto: select at random an image from either the set of target images or aset of non-target images; map the image to a probability score using thestyle similarity estimator, wherein the probability score indicates aprobability of the image belonging to the set of target images;calculate a loss based on the probability score; and update parametersof the style similarity estimator to reduce the loss using a gradientdescent algorithm.
 8. The image processing system of claim 7, whereinthe processor, when executing the instructions, to train the styletransfer network using the set of medical images, the set of targetimages, the clinical quality estimator, the style similarity estimator,and the image content regularizer, to produce the trained style transfernetwork is configured to: generate a set of style transferred medicalimages from the set of medical images using the style transfer network;determine a clinical quality loss by comparing the set of styletransferred medical images to the set of medical images using theclinical quality estimator; determine a similarity loss for the set ofstyle transferred medical images using the style similarity estimator;determine a content loss by comparing the set of style transferredmedical images to the set of medical images using the image contentregularizer; aggregate the clinical quality loss, the similarity lossand the content loss to produce a cumulative loss; and adjust one ormore parameters of the style transfer network based on the cumulativeloss.
 9. The image processing system of claim 8, wherein the processor,when executing the instructions, to adjust the one or more parameters ofthe style transfer network based on the cumulative loss is configuredto: backpropagate the cumulative loss through the clinical qualityestimator, the style similarity estimator, the image contentregularizer, and the style transfer network, to produce a backpropagatedcumulative loss; and update the one or more parameters of the styletransfer network based on the backpropagated cumulative loss using agradient descent algorithm.
 10. The image processing system of claim 8,wherein the processor, when executing the instructions, is furtherconfigured to: respond to the cumulative loss decreasing to below acumulative loss threshold by: storing the style transfer network in alocation of non-transitory memory, along with one or more pieces of metadata identifying the clinical quality estimator, the style similarityestimator, and the image content regularizer.
 11. The image processingsystem of claim 8, wherein the processor, when executing theinstructions, to aggregate the clinical quality loss, the similarityloss, and the content loss to produce the cumulative loss is configuredto: multiply the clinical quality loss by a first weight to produce aweighted clinical quality loss; multiply the similarity loss by a secondweight to produce a weighted similarity loss; multiply the content lossby a third weight to produce a weighted image content loss; and sum theweighted clinical quality loss, the weighted similarity loss, and theweighted content loss to produce the cumulative loss.
 12. The imageprocessing system of claim 8, wherein the clinical quality estimatorcomprises a pre-trained neural network, and wherein the processor, whenexecuting the instructions, to determine the clinical quality loss bycomparing the set of style transferred medical images to the set ofmedical images using the clinical quality estimator is configured to:map the set of style transferred medical images to a first plurality ofclinical quality scores using the clinical quality estimator; map theset of medical images to a second plurality of clinical quality scoresusing the clinical quality estimator; determine pair-wise differencesbetween the first plurality of clinical quality scores and the secondplurality of clinical quality scores to produce a plurality of clinicalquality losses; and sum an absolute value of each of the plurality ofclinical quality losses to produce the clinical quality loss.
 13. One ormore non-transitory computer-readable media encoding one or moreprocessor-executable routines, wherein the one or more routines, whenexecuted by a processor, cause acts to be performed comprising:selecting a set of medical images, wherein the set of medical images arein a first style; selecting a set of target images, wherein the set oftarget images are in a second style, and wherein the set of medicalimages and the set of target images are unpaired; selecting a clinicalquality estimator; selecting a style similarity estimator; selecting animage content regularizer; training a style transfer network using theset of medical images, the set of target images, the clinical qualityestimator, the style similarity estimator, and the image contentregularizer, to produce a trained style transfer network; receiving amedical image, wherein the medical image is in the first style; mappingthe medical image to a style transferred medical image using the trainedstyle transfer network, wherein the style transferred medical image isin the second style; displaying the style transferred medical image viaa display device; and wherein training the style transfer network usingthe set of medical images, the set of target images, the clinicalquality estimator, the style similarity estimator, and the image contentregularizer, to produce the trained style transfer network comprisestraining the style similarity estimator using the set of target images;and wherein the style similarity estimator comprises a neural network,and wherein the one or more routines, when executed by the processor,cause acts to be performed comprising: selecting at random an image fromeither the set of target images or a set of non-target images; mappingthe image to a probability score using the style similarity estimator,wherein the probability score indicates a probability of the imagebelonging to the set of target images; calculating a loss based on theprobability score; and updating parameters of the style similarityestimator to reduce the loss using a gradient descent algorithm.
 14. Theone or more non-transitory computer-readable media of claim 13, whereintraining the style transfer network using the set of medical images, theset of target images, the clinical quality estimator, the stylesimilarity estimator, and the image content regularizer, to produce thetrained style transfer network comprises: generating a set of styletransferred medical images from the set of medical images using thestyle transfer network; determining a clinical quality loss by comparingthe set of style transferred medical images to the set of medical imagesusing the clinical quality estimator; determining a similarity loss forthe set of style transferred medical images using the style similarityestimator; determining a content loss by comparing the set of styletransferred medical images to the set of medical images using the imagecontent regularizer; aggregating the clinical quality loss, thesimilarity loss and the content loss to produce a cumulative loss; andadjusting one or more parameters of the style transfer network based onthe cumulative loss.
 15. The one or more non-transitorycomputer-readable media of claim 14, wherein adjusting one or moreparameters of the style transfer network based on the cumulative losscomprises: backpropagating the cumulative loss through the clinicalquality estimator, the style similarity estimator, the image contentregularizer, and the style transfer network, to produce a backpropagatedcumulative loss; and updating the one or more parameters of the styletransfer network based on the backpropagated cumulative loss using agradient descent algorithm.
 16. The one or more non-transitorycomputer-readable media of claim 14, wherein the one or more routines,when executed by the processor, cause acts to be performed furthercomprising responding to the cumulative loss decreasing to below acumulative loss threshold by: storing the style transfer network in alocation of non-transitory memory, along with one or more pieces of metadata identifying the clinical quality estimator, the style similarityestimator, and the image content regularizer.
 17. The one or morenon-transitory computer-readable media of claim 14, wherein aggregatingthe clinical quality loss, the similarity loss, and the content loss toproduce the cumulative loss comprises: multiplying the clinical qualityloss by a first weight to produce a weighted clinical quality loss;multiplying the similarity loss by a second weight to produce a weightedsimilarity loss; multiplying the content loss by a third weight toproduce a weighted image content loss; and summing the weighted clinicalquality loss, the weighted similarity loss, and the weighted contentloss to produce the cumulative loss.
 18. The one or more non-transitorycomputer-readable media of claim 14, wherein the clinical qualityestimator comprises a pre-trained neural network, and whereindetermining the clinical quality loss by comparing the set of styletransferred medical images to the set of medical images using theclinical quality estimator comprises: mapping the set of styletransferred medical images to a first plurality of clinical qualityscores using the clinical quality estimator; mapping the set of medicalimages to a second plurality of clinical quality scores using theclinical quality estimator; determining pair-wise differences betweenthe first plurality of clinical quality scores and the second pluralityof clinical quality scores to produce a plurality of clinical qualitylosses; and summing an absolute value of each of the plurality ofclinical quality losses to produce the clinical quality loss.